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ABSTRACT 

A general 'approach to the :onstruction of acceleration of 
convergence methods for vector sequences is proposed. Using 
this approach one can generate some k:;oun methods and some 
neu ones. In this talk ue shall concentrate on one neu 
method, uhich turns out to be a simplified version of 
minimal polynomial extrapolation. U’e analyze the 
convergence of this method and shou that it is especially 
suitable for accelerating the convergence of vector 
sequences that are obtained uhen one solves linear systems 
of equations iteratively. 
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1 . INTRODUCTION 




Recently Smith and Ford 1 1 ] have reviaued and taatad 
numerically four methods of convergence acceleration for 
sequences of vectors. These methods are the minimal 
polynomial extrapolation (nPE)> the reduced rank 
extrapolation (RRE)» and the scalar and vector epsilon 
algorithms. One of the conclusions of this report* based on 
numerical experimentation* is that the MPE and the RRE have 
about the same properties, and in general* have better 
convergance than the others. 

The purpose of the present uork is to develop a general 
frameuork within which one can define a large class of 
vector acceleration methods* the MPE and the RPE being in 
this class. Me shall motivate this development in the way 
Shanks ( 3 ) motivated his development of the 
ek~trarisformation for scalar sequences. 


Shanks starts with a scalar sequence Sm* m = 0* 4* . . 
. , that has the property 


(1.1) Sm ~ S ♦ ^ Otj x7 as m^«* 

i=l 

where S is lim Sm if all | X i | < 1 ; otherwise S is called the 
m*«> 

"anti-limit" of the sequence Sm*. m = 0* 1, . . . As one 
way of approximating S* Shanks proposes to solve the 2k-t-l 
nonlinear equations 


k 

(1.2) Sm=Snk+^ aix7»n<m< n^2k 

* 

i=l 

for Sn^k* tt||X|* i = 1* . . . * k. By solving the system 
in (1.2)> Shanks has expressed Sn^x as the quotient of two 
determinants as follows: 
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Sn'^k 

ASn*k 
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(1.3) Sn^k 


Sn 

ASn 


Sn*i . . . 

ASn*i . . . 


^n*K-1 ASn^k 


AS|>*ak-i 


1 

ASn 


1 

ASn ►i 


. . I 
• . ASn* 


n'^k 


ASn+k”l ASn+k • • * ASn + Jk-i 


Tuo equivalent formulationF fcllou from (l.S): 

a) It is clear from (1.3) the; Sn^k along with the k 
parameters i > 0» 1, . . . , k~l. selves the set of k4-l 
linear equations 

k-1 

(1.^) Sm “ Sn^k * Pi ASm + 1 * n £ m £ n+k. 

i=0 

By taking the differences of the equations in (l.^)> ue can 
see that the Pi satisfy 
k-1 

(1.5) ASm = Pi A*Sm*i. n < m < r.+k-l, 

i=0 

and that once the have been determined* Sn k can be 

t 

computed from one of the equations in (1.4). 

b) From (1.4) it follows that Sn k along with the 

t 

parameters >i*i=0* . . . ,k* satisfy the equations 
k 

(1.6) Sn^i< “ 1 1 Sin^ i> n S m S n+k» 

i=0 

subject to 

k 

(1.7) ^ Y, « 1. 
i=0 

By taking differences of the equations in (1.6)* ue can see 
that the Yi satisfy 
k 

(1.8) 0 « ^ Yi Asm'll* n < m < n+k-1* 

i«0 
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subject to (1.7). Once the Yi have been determined. Sn^a 
can be computed from one of the equations in (1.6). 

Furthermore, if Yk * 0. then (1.7) and (1.8) are equivalent 
to 

k-1 

(1.9) ^ * -ASm+k. n i m < n+k-1. 

i=0 

uhere 

(1.10) TTi ' 0 < i < k-1, Ck51, 

OF POOR QUALITY 

Cj 

j=o 

k 

provided ^ Cj # 0. 

j = 0 

It has been proved by Uynn ( 4 ] . that Sn k . uhen applied 

t 

to sequences Sm, m = 0, 1, . . . , that are of the form 
given in (l.l). converges to S as n-»« (k fixed), under 
certain conditions on the Xi, faster than Sn itself. Uynn 
actually gives rates of convergence, and also analyses the 
stability properties of the approximations S„ k» uhen errors 

f 

are introduced in the Sm. 

In the next section ue shall extend the tuo 
formulations (a and b) given above for deriving Shanks' 
transformations, to vector sequences, and shall obtain a 
class of vector acceleration methods that includes some of 
the knoun methods as well as some neu ones. In Section 3 ue 
shall analyze one of these methods, subject to an assumption 
on the vector sequence analogous to (1,1). and shall prove 
that it accelerates the convergence of the sequence, and 
give an actual rate of convergence for it. 

2. DERIVATION OF VECTOR ACCELERATION HETHODS 

Let us nou consider a sequence of vectors, x . m » 0. 

1. . . . . in a general space B. satisfying 
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(2.1) !(«-•♦ ^ V| x7 at 

i«l 

uhara a and tha V( ara vactora* and tha X| ara aealara. 

Again S ia tha limit or anti-limit of thia aaquanoa* 
dapending cn tha X|. A vary aimpla and practical axampla of 
auch a aequenca ia that produced by a matrix iterative 
technique for solving tha aquation 

(2.2) K B Ax « bt 

uhere A ia an n x M matrix, and b and x are n-dimenaional 
column vectors. If s is tha solution to (2.2) and for given 
xo» the vectors Xm are generated by 

(2.*‘. Xm^l ° AXm *b> m>0. 1 

then 

n 

r 

(2. A) Xm*a* > V|Xi>msO. !»...# 

i*=l 

uhere X| and V| are the eigenvalues and corresponding 
eigenvectors respectively, of the matrix A, assuming that A 
has precisely M eigenvectors. The condition stated in (2.1) 
is analogous to that stated in (1.1) for scalar sequences. 
Since the Shanks' transformation accelerates the convergence 
of scalar sequences satisfying (1.1), ue expect that its 
extensions to the vector case, through the formulations a) 
and b) in the previous section, uill also produce 
acceleration of convergence for vector sequences satisfying 
( 2 . 1 ). 

The extensions of the two formulations can be achieved 
as follous; 

a) Let us start by writing (1.3)-(1.^) in terms of the 
vector sequence. We have 

k-1 

(2.5) ^ ^1 Ax„+, 

i«0 

with $i obtained from the overdetermined system 
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k 

(2.6) AKm ” ^ A^Xm^ir n £ m S n^k'l 

i=0 

by some technique. 

b) If ue write (1.6)-(1.10) in terms of the vector 
sequence* ue find 

k 

(2.7) S„ k ' 

i^O 

where Ti are obtained from the overdetermined system 



k-1 




(2.8) 

I 

AXm+l 

= - AXm+k. n 

< m S n+k-1* 


i=0 




and 


Ci 



(2.9) 


0 < i < k-1* 

Ck^l » 


IT 


!=• 

j=0 

k 

provided cj # 0 . 

3=0 

Ue see that for both approaches* ue need to "solve" an 
overdetermined and* in general* inconsistent system of 
equations of the form 

k-1 

(2.10) ^ d| Um+j * Um* n < m < n+k-1, 

i=0 

where uj and U| are members of the space B and di are 
unknown scalars. If r. the dimension of B* is greater than 
or equal to k* then even one of the equations in (2.10) 
gives rise to an overdetermined system. Ue can* however* 
propose various ways for obtaining a set of di* that 
"solves" (2.10) in some sense. In what follows* we give 
three such methods* with the understanding that other 
methods can also be proposed. 
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1) Assuming n 2 k» 
(2.10) uith m«n only] 


( 2 . 11 ) 


k-1 

^ di u„+i 

i*0 


solve the overdetermined system leq. 
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by least squares. For finite dimensional spaces B> this 
method gives us RRE for approach a)» and it gives nPE for 
approach b). 


2) Assuming r > k# solve the set of k equations 

k-1 

(2.12) ^ di Qj (Wn+l) “ Q| (Un)» j « 0, 1» . . . , k-1, 
i=0 

uhere Qj are linearly independent operators in the dual 
space of B. For Hilbert spaces, ue can take Qi(y) ^ (qj,y), 
uhere qj are vectors and ( . , . ) is an inner product. 

3) Solve the set of equations 

k-1 

(2.13) ^ di Q(Um+i) = Q(um), n < m < n+k-1, 
i=0 

uhere Q is an operator in the dual space of B. This method 
is similar to that introduced by Brezinski [21. 


3. A CONVERGENCE RESULT 


Ue nou give a convergence result for the method that is 
obtained from approach b) using the procedure in 2) in the 
previous section. Ue shall state the result, but leave out 
its proof. The proof, along uith several other neu results 
uill be included in the final version of this paper. For 
simplicity ue shall assume that B is finite dimensional. 

Theorem: Let the sequence Km, m > 0, 1, . . . be as 

in (2.3)-(2.4). Let S.- k be obtained from (2.7)-(2.9) uith 

t 

the Cl obtained by solving the linear system 


k-1 

^ Cl Qj (Axh-^i) “ -Qj(AKn*k), j ■ t, 1, . . . , k-1 
i*0 
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uhar* Qj ar* k linaarly indapandant oparatora in tha dual 
apaea of B. (For aMampla. Qi(y) ■ (j^l)at oomponant of y, 

3 ■ 0, I k-1.) If Ui| I |Xsl i . . . 2 |XkI > 

IXk^iI > IXk^iI 2 . . » than* In ganaral, 

(3.1) Snk-S«0 •* 

Recall that Kn - S * 0(x") as n«-». Tharafora. 
uhanaver the conditions of the theorem above are satisfied r 
than Sn k converges to S faster than Xn uhen tha latter 
converges! and if |Xk'*’iI < 1> then Sn^k converges to S even 
if Kn may diverge. 
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